Aiming at the problems of complex retinal image processing, poor generalization and lack of complete automatic recognition system, a complete retinal image automatic recognition system based on deep neural network was proposed. Firstly, the image was denoised, normalized, and data preprocessed. Then, a compact neural network model named CompactNet was designed. The structure parameters of CompactNet were inherited from AlexNet. The deep network parameters were adjusted adaptively based on the training data. Finally, the performance experiments were conducted on different training methods and various network structures. The experimental results demonstrate that the fine-tuning method of CompactNet is better than the traditional network training method, the classification index can reach 0.87, 0.27 higher than the traditional direct training. By comparing LeNet, AlexNet and CompactNet, CompactNet network model has the highest classification accuracy, and the necessity of preprocessing methods such as data amplification is confirmed by experiments.